<html lang="en-US">

<head>
    
<title>在 Kubernetes 中部署应用程序 - 花落雨忧</title>

<meta property="og:title" content="在 Kubernetes 中部署应用程序 - 花落雨忧">



    



    
    <meta property="description" content="在完成了本地 Kubernetes 的快速搭建（基于 Docker）后，我们已经可以正式的使用它了。对于我们平时最常见的需求，那就是往 Kubernetes 里部署应用程序，如果你没有看过 Kubernetes 相关的知识，这时候你可能会六神无主，但问题不大，我们就可以使用最经典的 Nginx 来小试身手。
[&amp;hellip;] 创建 nginx-deployment.yaml 文件： &amp;hellip;">
    <meta property="og:description" content="在完成了本地 Kubernetes 的快速搭建（基于 Docker）后，我们已经可以正式的使用它了。对于我们平时最常见的需求，那就是往 Kubernetes 里部署应用程序，如果你没有看过 Kubernetes 相关的知识，这时候你可能会六神无主，但问题不大，我们就可以使用最经典的 Nginx 来小试身手。
[&amp;hellip;] 创建 nginx-deployment.yaml 文件： &amp;hellip;">
    






<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1">

<link rel="shortcut icon" href="https://www.lican.site/logo/logo.png" type="image/x-icon" />



<link rel="stylesheet" href="/css/style.min.css" />

<link rel="stylesheet" href="/css/reset.min.css" />




<script src="https://www.lican.site/js/highlight.min.js"></script>

<script>
hljs.configure({ ignoreUnescapedHTML: true })
hljs.highlightAll();
</script>


<script src="https://www.lican.site/js/jquery.min.js"></script>




<link href="https://www.lican.site/css/hugo-code.min.css" rel="stylesheet" />



    <style>
        .post-content img {
            max-width: 400px;
        }
    </style>
</head>

<body id="period" class="home blog">
    <a class="skip-content" href="#main">Press "Enter" to skip to content</a>
    <div id="overflow-container" class="overflow-container">
        <header class="site-header" id="site-header" role="banner">
    <div class="max-width">
        <div id="title-container" class="title-container">
            <div id="site-title" class="site-title"><a href="/">花落雨忧</a></div>
            <p class="tagline">思所及 力所行 方为真.</p>
        </div>
        
        <div id="menu-primary-container" class="menu-primary-container">
            <div class="icon-container">
            </div>
            <div id="menu-primary" class="menu-container menu-primary" role="navigation">
                <nav class="menu">
                    <ul id="menu-primary-items" class="menu-primary-items">
                        
                        
                        <li id="menu-item-0"
                            class="menu-item menu-item-type-custom menu-item-object-custom ">
                            <a href="/posts" aria-current="page" tabindex="0">首页</a></li>
                        
                        <li id="menu-item-1"
                            class="menu-item menu-item-type-custom menu-item-object-custom ">
                            <a href="/tech/" aria-current="page" tabindex="1">技术文档</a></li>
                        
                        <li id="menu-item-2"
                            class="menu-item menu-item-type-custom menu-item-object-custom ">
                            <a href="/article/" aria-current="page" tabindex="2">文章</a></li>
                        
                        <li id="menu-item-3"
                            class="menu-item menu-item-type-custom menu-item-object-custom ">
                            <a href="/project/" aria-current="page" tabindex="3">项目</a></li>
                        
                        <li id="menu-item-4"
                            class="menu-item menu-item-type-custom menu-item-object-custom ">
                            <a href="/about/" aria-current="page" tabindex="4">关于</a></li>
                        
                    </ul>
                </nav>
            </div>
        </div>
    </div>
</header>
        <div id="primary-container" class="primary-container">
            <div class="max-width">
                <section id="main" class="main" role="main">
                    <div id="loop-container" class="loop-container">
                        <div
                            class="post type-post status-publish format-standard hentry entry">
                            <article>
                                <div class="post-container">
                                    <div class="post-header">
                                        <h2 class="post-title">
                                            <a href="/posts/kubernetes/2020-05-03-deployment/">在 Kubernetes 中部署应用程序</a>
                                        </h2>
                                        
                                        <div class="post-byline">Published on
                                            <a class="date" href="javascript:;">2020/05/03</a>
                                            
                                            
                                            
                                            
                                            
                                        
                                    </div>
                                    <div class="post-content">
                                        <p>在完成了本地 Kubernetes 的快速搭建（基于 Docker）后，我们已经可以正式的使用它了。对于我们平时最常见的需求，那就是往 Kubernetes 里部署应用程序，如果你没有看过 Kubernetes 相关的知识，这时候你可能会六神无主，但问题不大，我们就可以使用最经典的 Nginx 来小试身手。</p>
<h2 id="创建-deployment">创建 Deployment</h2>
<p>创建 nginx-deployment.yaml 文件：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-yaml" data-lang="yaml"><span style="color:#66d9ef">apiVersion</span>: apps/v1
<span style="color:#66d9ef">kind</span>: Deployment
<span style="color:#66d9ef">metadata</span>:
  <span style="color:#66d9ef">name</span>: nginx-deployment
  <span style="color:#66d9ef">labels</span>:
    <span style="color:#66d9ef">app</span>: nginx
<span style="color:#66d9ef">spec</span>:
  <span style="color:#66d9ef">replicas</span>: <span style="color:#ae81ff">2</span>
  <span style="color:#66d9ef">selector</span>:
    <span style="color:#66d9ef">matchLabels</span>:
      <span style="color:#66d9ef">app</span>: nginx
  <span style="color:#66d9ef">template</span>:
    <span style="color:#66d9ef">metadata</span>:
      <span style="color:#66d9ef">labels</span>:
        <span style="color:#66d9ef">app</span>: nginx
    <span style="color:#66d9ef">spec</span>:
      <span style="color:#66d9ef">containers</span>:
      - <span style="color:#66d9ef">name</span>: nginx
        <span style="color:#66d9ef">image</span>: nginx:<span style="color:#ae81ff">1.18.0</span>
</code></pre></div><p>应用 nginx-deployment.yaml 文件：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl apply -f nginx-deployment.yaml
deployment.apps/nginx-deployment created
</code></pre></div><h2 id="查看运行状态">查看运行状态</h2>
<p>查看 Pod 运行情况：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
nginx-deployment-9fbc65d67-9j68x   1/1     Running   <span style="color:#ae81ff">0</span>          1m
nginx-deployment-9fbc65d67-nwbhj   1/1     Running   <span style="color:#ae81ff">0</span>          1m
</code></pre></div><p>查看 Deployment 部署情况：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl get deployment
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   2/2     <span style="color:#ae81ff">2</span>            <span style="color:#ae81ff">2</span>           29m
</code></pre></div><p>我们也可以通过 describe 命令进行查看</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl describe pod nginx-deployment-9fbc65d67-9j68x
Name:           nginx-deployment-9fbc65d67-9j68x
Namespace:      default
Priority:       <span style="color:#ae81ff">0</span>
Node:           docker-desktop/192.168.65.3
Start Time:     Fri, <span style="color:#ae81ff">01</span> May <span style="color:#ae81ff">2020</span> 17:36:12 +0800
Labels:         app<span style="color:#f92672">=</span>nginx
                pod-template-hash<span style="color:#f92672">=</span>9fbc65d67
...
Events:
  Type    Reason     Age   From                     Message
  ----    ------     ----  ----                     -------
  Normal  Scheduled  45m   default-scheduler        Successfully assigned default/nginx-deployment-9fbc65d67-9j68x to docker-desktop
  Normal  Pulling    45m   kubelet, docker-desktop  Pulling image <span style="color:#e6db74">&#34;nginx:1.18.0&#34;</span>
  Normal  Pulled     44m   kubelet, docker-desktop  Successfully pulled image <span style="color:#e6db74">&#34;nginx:1.18.0&#34;</span>
  Normal  Created    44m   kubelet, docker-desktop  Created container nginx
  Normal  Started    44m   kubelet, docker-desktop  Started container nginx
</code></pre></div><h2 id="查看-dashboard">查看 Dashboard</h2>
<p>在应用了 Nginx 的 Deployment 后，我们可以查看上一章节中我们所搭建的 Dashboard：</p>
<p><img src="https://image.eddycjy.com/8e08a407d9333759e99a5f09f18e6e8c.jpg" alt="image"></p>
<p>可能你在想，我只是执行了一条命令，怎么就把 Nginx 跑起来了，这时候你可以去查看容器组中的事件，就能够看到这个容器在运行时做涉及到的事件：</p>
<p><img src="https://image.eddycjy.com/8d96979504a96c5d5b60fad3eeb35060.jpg" alt="image"></p>
<h2 id="部署-nginx">部署 Nginx</h2>
<h3 id="创建-nginx-service">创建 Nginx Service</h3>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx
spec:
  selector:
    app: nginx
  ports:
  - name: nginx-port
    protocol: TCP
    port: <span style="color:#ae81ff">80</span>
</code></pre></div><p>应用 nginx-service.yaml 文件：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl apply -f nginx-service.yaml
</code></pre></div><p>查看应用的运行情况：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ kubectl get services -o wide
</code></pre></div><p>但这时候是无法访问到 Nginx 的，我们可以通过 Kubernetes 的 NodePort 的方式对外提供访问：</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  labels:
    app: nginx
spec:
  selector:
    app: nginx
  ports:
  - name: nginx-port
    protocol: TCP
    port: 80
    nodePort: 30001
    targetPort: 80
  type: NodePort
</code></pre><p>然后再进行访问：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ curl http://127.0.0.1:30001
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Welcome to nginx!&lt;/title&gt;
...


</code></pre></div><p>至此我们已经打通了和 Nginx 之间的访问。</p>
<h2 id="部署-go-程序">部署 Go 程序</h2>
<p>在部署环境中常常需要将应用程序部署上去，然后对外进行提供服务，我们模拟一个 Go 程序：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-go" data-lang="go"><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">main</span>() {
    <span style="color:#a6e22e">r</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">gin</span>.<span style="color:#a6e22e">Default</span>()
    <span style="color:#a6e22e">r</span>.<span style="color:#a6e22e">GET</span>(<span style="color:#e6db74">&#34;/ping&#34;</span>, <span style="color:#66d9ef">func</span>(<span style="color:#a6e22e">c</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">gin</span>.<span style="color:#a6e22e">Context</span>) {
        <span style="color:#a6e22e">c</span>.<span style="color:#a6e22e">String</span>(<span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">StatusOK</span>, <span style="color:#e6db74">&#34;pong&#34;</span>)
    })
    <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">r</span>.<span style="color:#a6e22e">Run</span>(<span style="color:#e6db74">&#34;:9001&#34;</span>)
    <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
        <span style="color:#a6e22e">log</span>.<span style="color:#a6e22e">Fatalf</span>(<span style="color:#e6db74">&#34;r.Run err: %v&#34;</span>, <span style="color:#a6e22e">err</span>)
    }
}
</code></pre></div><h3 id="编写和编译-dockerfile">编写和编译 Dockerfile</h3>
<p>在项目根目录创建 Dockerfile 文件，进行编写：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">FROM golang:latest

ENV GOPROXY https://goproxy.cn,direct
WORKDIR $GOPATH/src/github.com/eddycjy/awesome-project
COPY . $GOPATH/src/github.com/eddycjy/awesome-project
RUN go build .

EXPOSE <span style="color:#ae81ff">8000</span>
ENTRYPOINT <span style="color:#f92672">[</span><span style="color:#e6db74">&#34;./awesome-project&#34;</span><span style="color:#f92672">]</span>
</code></pre></div><p>编译并打标签：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ docker build -t eddycjy/awesome-project:v0.0.1 .
...
Successfully built b53cef4d2967
Successfully tagged eddycjy/awesome-project:v0.0.1
</code></pre></div><p>验证打包进 Docker 中的程序是否正常运行：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ docker run -p 10001:9001 awesome-project
<span style="color:#f92672">[</span>GIN-debug<span style="color:#f92672">]</span> GET    /ping                     --&gt; main.main.func1 <span style="color:#f92672">(</span><span style="color:#ae81ff">3</span> handlers<span style="color:#f92672">)</span>
<span style="color:#f92672">[</span>GIN-debug<span style="color:#f92672">]</span> Listening and serving HTTP on :9001
<span style="color:#f92672">[</span>GIN<span style="color:#f92672">]</span> 2020/05/03 - 01:51:40 | <span style="color:#ae81ff">200</span> |        16.9µs |      172.17.0.1 | GET      <span style="color:#e6db74">&#34;/ping&#34;</span>
</code></pre></div><h3 id="上传到-dockerhub">上传到 Dockerhub</h3>
<p>登陆并推送镜像到 Dockerhub：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ docker login  
$ docker push eddycjy/awesome-project:v0.0.1      
The push refers to repository <span style="color:#f92672">[</span>docker.io/eddycjy/awesome-project<span style="color:#f92672">]</span>
8192ac09ffeb: Pushed 
9eb17b90d619: Pushed 
b04d698ea69d: Pushed 
31561785c3fc: Mounted from library/golang 
4486631650dc: Mounted from library/golang 
5e28718a7d23: Mounted from library/golang 
ea1227feeccb: Mounted from library/golang 
9cae1895156d: Mounted from library/golang 
52dba9daa22c: Mounted from library/golang 
78c1b9419976: Mounted from library/golang 
v0.0.1: digest: sha256:a1ef61e899db75eb2171652356be15559f1991b94a971306fb79ceccea8dd515 size: <span style="color:#ae81ff">2422</span>
</code></pre></div><p>这时候你在 hub.docker.com 上就能看的你刚刚所上传的镜像内容：</p>
<p><img src="https://image.eddycjy.com/605dca784a33f0466c655d4418818154.jpg" alt="image"></p>
<h2 id="编写-kubernetes-配置">编写 Kubernetes 配置</h2>
<p>接下来我们需要针对刚刚所打包的 Go 程序创建 Deployment，编写 go-deployment.yaml 文件：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: awesome-project
  labels:
    app: awesome-project
spec:
  replicas: <span style="color:#ae81ff">2</span>
  selector:
    matchLabels:
      app: awesome-project
  template:
    metadata:
      labels:
        app: awesome-project
    spec:
      containers:
      - name: awesome-project
        image: eddycjy/awesome-project:v0.0.1
</code></pre></div><p>创建 Service，编写 go-service.yaml：</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: awesome-project-svc
  labels:
    app: awesome-project
spec:
  ports:
  - port: 9001
  type: ClusterIP
  selector:
    app: awesome-project
</code></pre><h2 id="部署-ingress">部署 Ingress</h2>
<h3 id="ingress-controller">Ingress Controller</h3>
<p>我们采用 Docker for Mac 特定提供的 Ingress Controller 部署脚本：</p>
<pre><code>$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-0.32.0/deploy/static/provider/cloud/deploy.yaml
</code></pre><p>其会所有命名空间监视 Ingress 对象，并配置 RBAC 权限，否则你有可能会遇到 403 Forbidden 的问题。</p>
<h3 id="nginx-ingress">Nginx Ingress</h3>
<p>在完成了 Ingress Controller 等相关部署后，我们可以正式的部署属于自己业务的 Nginx Ingress 对象：</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/use-regex: &quot;true&quot;
spec:
  rules:
  - host: website-ingress.local
    http:
      paths:
      - backend:
          serviceName: awesome-project-svc
          servicePort: 9001
</code></pre><p><code>kubectl apply -f</code> 应用刚刚所编写的配置文件，然后查看运行情况：</p>
<pre><code>$ kubectl get ingresses.
NAME           HOSTS                   ADDRESS     PORTS   AGE
test-ingress   website-ingress.local   localhost   80      8h
</code></pre><p>如何发现 ADDRESS 为空，则存在问题，需要进行排查（可能性有很多）。在确定 ADDRESS 属性正常后，我们需要打开 <code>/etc/hosts</code> 并配置 HOST <code>127.0.0.1 awesome-project.local</code> ，并进行验证：</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">$ curl http://awesome-project.local/ping
pong
</code></pre></div><p>至此，我们完成了一个简单的 Go 程序的部署和外部调用。</p>
<h2 id="小结">小结</h2>
<p>在本章中，我们通过部署 Nginx、Ingress、Go 程序的方式，直接实践了 Kubernetes 的基本流程，达到了将自己的简单程序部署在 Kubernetes 的一个小目标，接下来在后续的章节中我们将进一步针对文中所使用到的相关属性和内容进行详细说明。</p>
<p>毕竟在实践过后，就要去了解为什么，这样子才能做到融会贯通。</p>


                                        
                                        
                                        
                                        <div class="rp4wp-related-posts">
                                            <h3>相关文章</h3>
                                            <ul>
                                                
                                                <li>
                                                    <div class="rp4wp-related-post-content">
                                                        <a
                                                            href="https://www.lican.site/posts/kubernetes/2020-05-01-install/">Kubernetes 本地快速启动（基于 Docker）</a>
                                                    </div>
                                                </li>
                                                
                                            </ul>
                                        </div>
                                        
                                        
                                    </div>

                                    
                                    
                                    

                                    
                                    <div class="post-meta">
                                        
                                        
                                        <div class="post-tags">
                                            <ul>
                                            
                                            <li>
                                                <a href="/tags/kubernetes" title="View all posts tagged match">kubernetes</a>
                                            </li>
                                            
                                            </ul>
                                        </div>
                                        
                                        
                                        <nav class="further-reading">
                                            
                                            <div class="previous">
                                                <span>&lt;&lt; Prev</span>
                                                <a href="https://www.lican.site/posts/kubernetes/2020-05-01-install/"
                                                    rel="prev">Kubernetes 本地快速启动（基于 Docker）</a> </div>
                                            
                                            
                                            <div class="next">
                                                <span>Next >></span>
                                                <a href="https://www.lican.site/posts/kubernetes/2020-05-10-api/">使用 Go 程序调用 Kubernetes API</a> 
                                            </div>
                                            
                                        </nav>
                                    </div>
                                    
                                    

                                    
                                    
                                    

                                </div>
                            </article>
                        </div>
                    </div>
                </section>
                <aside class="sidebar sidebar-primary" id="sidebar-primary" role="complementary">
    <h1 class="screen-reader-text">Sidebar</h1>
    

    
    
    <section id="text-2" class="widget widget_text">
        <div class="textwidget">
            
            <div id="profile">
                <div id="profile_picture"><img src="https://www.lican.site/logo/logo.png"></div>
                <div id="profile_intro">
                    <p><span class="name">Lican</span></p>
                    <p class="intro">全栈开发者，爱好造轮子。</p>
                </div>
            </div>
            
            <p>
                <script type="text/javascript">
                    (function ($) {
                        $(document).ready(function () {
                            var menuPrimaryContainer = $('#menu-primary-container');
                            var profile = $('#text-2');
                            $('#toggle-navigation').click(function () {
                                if (menuPrimaryContainer.hasClass('open')) {
                                    profile.removeClass('open');
                                } else {
                                    profile.addClass('open');
                                }
                            });
                        });
                    })(jQuery);
                </script>
            </p>
        </div>
    </section>
    
    
    
    
    
    <section id="text-5" class="widget widget_text">
        <h2 class="widget-title">开源项目</h2>
        <div class="textwidget">
            <div id="projects" style="line-height: 22px;">
                
                <a href="https://github.com/idoubi/gonews"
                    target="_blank">gonews</a>: &nbsp;Daily news for golang<br>
                
                <a href="https://github.com/idoubi/sql2struct"
                    target="_blank">sql2struct</a>: &nbsp;Generate go struct according to SQL<br>
                
                <a href="https://github.com/idoubi/goz"
                    target="_blank">goz</a>: &nbsp;Request library used in golang<br>
                
        </div>
    </section>
    
    

    
    
    
    
    

    
    
    

    
    
    

    
    
    
    
</aside>
            </div>
        </div>

        <footer id="site-footer" class="site-footer" role="contentinfo">
    <div class="max-width">
    </div>
    <div class="footer">
        <div id="footercontent">
            © lican.vip All rights reserved<br/>
            Built with Hugo Theme <a href="https://github.com/idoubi/hugo-theme-period" target="_blank">Period</a>
        </div>
    </div>
</footer>

<script>
    var _hmt = _hmt || [];
    (function() {
      var hm = document.createElement("script");
      hm.src = "https://hm.baidu.com/hm.js?e8351b6d4626d5881d439ea1f6184baa";
      var s = document.getElementsByTagName("script")[0]; 
      s.parentNode.insertBefore(hm, s);
    })();
</script>
    
    
    </div>
    
</body>

</html>